Feature Selection for Ordinal Text Classification1

Ordinal classification (also known as ordinal regression) is a supervised learning task that consists of estimating the rating of a data item on a fixed, discrete rating scale. This problem is receiving increased attention from the sentiment analysis / opinion mining 1This is a revised and substantially extended version of a paper appeared as (Baccianella et al., 2010). The order in which the authors are listed is purely alphabetical; each author has given an equally important contribution to this work. community, due to the importance of automatically rating large amounts of product review data in digital form. As in other supervised learning tasks such as binary or multiclass classification, feature selection is often needed in order to improve efficiency and to avoid overfitting. However, while feature selection has been extensively studied for other classification tasks, it has not for ordinal classification. In this paper we present six novel feature selection methods that we have specifically devised for ordinal classification, and test them on two datasets of product review data against three methods previously known from the literature, using two learning algorithms from the “support vector regression” tradition. The experimental results show that all six proposed metrics largely outperform all three baseline techniques (and are more stable than them by an order of magnitude), on both datasets and for both learning algorithms.